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Summary 

Onboard science data analysis will improve the capabilities of existing sensors and 
enable transformative new operational modes to address novel science issues. Two 
missions illustrate the trend: the Mars Exploration Rovers (MER) and the Earth Observer 
1 spacecraft (EO-1). MER software autonomously interprets collected science data to 
identify features of interest such as clouds and dust devils. It responds by prioritizing 
data for downlink to Earth [10]. The EO-1 orbiter recognizes transient events and 
anomalies such as volcanic activity and responds with folio wup measurements [12]. In 
general, onboard analysis can improve science yield by relieving constraints on time, 
bandwidth and power, and by responding automatically to events on short time scales. 

Autonomy will advance rapidly in coming years due to progress in the fields of 
artificial intelligence, robotics, computer vision, and machine learning. We recommend 
the panel support technology development to exploit these advances. Previously NASA 
programs such as ASTEP and AISRP have played a key role in advancing autonomous 
science data analysis. Growing resources for technology development will help planetary 
science benefit from advances in AI research. We also recommend recognizing 
autonomous onboard analysis in plans for future missions. Planning early for onboard 
analysis will ease infusion into future missions and enhance these instruments' science 
yield. This document summarizes the potential impact of some specific onboard data 
understanding systems. We will describe applications to rover, aerobot, and orbital 
platforms. 

Rover and Lander Science 

Automatic Target Detection and Response Improves Science Yield 

Thanks to improvements in rover mobility, onboard data understanding could benefit 
rover platforms for potential Mars Sample Return (MSR) or Mars Astrobiology Field 
Laboratory (AFL) missions. Recent decades have seen significant progress in daily 
traverse ranges. Terrain analysis, visual odometry, and path planning algorithms permit 
Mars Exploration Rover (MER) spacecraft to navigate tens of meters autonomously [30, 
7]. Terrestrial tests demonstrate kilometer-scale traverses through Mars analog terrain 
[49, 5, 18, 19]. Here rovers travel beyond the local visible horizon to survey large areas 
and visit multiple geologic units in a single command cycle. 

Onboard science data analysis can exploit these kilometer-scale operations by 
identifying new science targets near the rover path and executing opportunistic followup 
measurements [38, 11, 40]. This enables fast traverse without sacrificing our 
understanding of the terrain visited en route. Over-the-horizon operational modes could 
benefit potential MSR and AFL objectives by quickly characterizing regional diversity in 
composition, morphology, and sedimentology over large geographic areas. These 
approaches hold promise for astrobiology investigations since evidence of potential 
habitats may be geographically isolated and invisible from orbit [5 , 48] . 

Rovers can recognize science features for opportunistic sensor deployment [22, 38, 
10]. Recent work has demonstrated fully automatic onboard target detection algorithms 
that recognize geomorphological features and anomalies [17, 43, 50]. Pattern recognition 
has automated a growing portfolio planetary geology tasks. These include detecting and 



classifying rocks [22, 17, 6], identifying anomalous outcrop [22, 38, 43], and 
characterizing sediment [44] . 

After its expected upload to the Mars Exploration Rovers, the AEGIS system will 
apply these techniques to support the Meridiani cobble campaign (Figure 1 A,B). AEGIS 
identifies targets in MER NavCam images and matches them against desired feature 
profiles provided by scientists. Scientists can configure AEGIS to target specific 
features, for example making it seek eccentric rocks with high albedo. The system 
responds with high-resolution subframed PanCam images of the most promising targets. 

AEGIS typifies autonomous science systems that favor a particular class of important 
features or minerals that are set by scientists in advance [40, 4, 50, 17]. However, 
scientists cannot always anticipate the features that the rover might encounter. 
Alternative modes include representative sampling that automatically categorizes the 
data into archetypal classes and sends back salient examples of each class [9, 25]. 
Alternatively, anomaly detection seeks out statistical outlier features that are unlike any 
yet encountered [9, 43]. Statistical and machine learning methods for anomaly detection 
are common in industrial applications [1, 27], and these techniques transfer naturally to 
autonomous science target selection. 

Followup measurements can take the form of high-resolution full-filtered images 
[17], remote reflectance spectroscopy, or thermal emission spectroscopy. Field tests with 
rover-mounted spectrometers have demonstrated the collection of dozens of spectra per 
command cycle from rocks that were detected with no human intervention [6] . Contact 
spectroscopy is also possible thanks to new single-cycle instrument placement techniques 
that can place instruments with minimal human guidance [35, 36, 50]. The ability to 
detect new targets, collect spectroscopy on the fly, and return to the scripted mission plan 
could be a transformative new option for long-range travel in potential MSR and AFL 
missions. 

Finally, research has also demonstrated onboard planning and scheduling systems that 
can make context-sensitive followup decisions to balance the science value of the new 
data against mission objectives. This permits opportunistic reaction to targets of 
opportunity while simultaneously considering immediate and long-term goals related to 
resource usage and end-of-day position [41 , 10, 50]. 
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Figure 1: Automatic Onboard Image Analysis for MER Platforms. A) Automatic target detection 
using the AEGIS system. B) Target detail. C) Dust devil tracking across four NavCam frames. D) 
Summary data product, ~3KB size. E) Thumbnail images of dust devils. Images courtesy 
NASA/JPL/Cornell. 



Selective Data Return Alleviates Bandwidth Constraints 

Thanks to improvements in sensor resolution, science data volumes already outpace 
our ability to communicate the relevant measurements to Earth. There is a compelling 
case for rover data products such as high-resolution cameras and video that have even 
larger data rates. These products can benefit both science (to resolve fine features) and 
public outreach (to better realize a virtual "presence" through high-definition images and 
video). It is likely that we are entering a new phase of planetary exploration in which 
communications bandwidth becomes a key science bottleneck. Future rovers must 
reduce the vast data set of potential collected data to a manageable volume for 
transmission over the Deep Space Network. 

Onboard analysis can relieve bandwidth constraints in several ways. First, spacecraft 
can collect large datasets that are subsampled using feature detection and prioritization to 
ensure that the most informative data products are transmitted. This can effectively 
multiply the science value of long traverses, enabling true survey modes that return 
representative images from geologic boundaries [42] or the main terrain types 
encountered during a traverse [11]. In addition, onboard data understanding permits 
spacecraft to summarize observations using more abstract descriptions. For example, 
mineral detection based on machine learning can automatically interpret and summarize 
spectral measurements [4, 21, 34]. Adaptive image compression can preserve fidelity in 
areas of high scientific interest [47] . 

Selective data return can unlock the temporal domain by recognizing and reacting to 
changes in the rover's environment. This is particularly poignant in the case of dust 
devils since they are rare and difficult to capture by chance. Recently, the WATCH 
system aboard the Mars Exploration Rovers demonstrated automatic selective data return 
that monitored image sequences to detect dust devils and transmit these images. Figure 1 
(C,D,E) shows the result of dust devil tracking on a series of four NavCam images. 
Automatic detection produces a summary bitmask that reduces bandwidth requirements 
by orders of magnitude, and thumbnail images can provide validation images of the dust 
devil with little additional cost [11]. The WATCH system has already run successfully 
on Mars for several years, enabling atmospheric studies that would have been impossible 
otherwise due to onerous bandwidth requirements. 

Selective data return can benefit future missions by increasing the effective science 
yield of high rate image sequences. Selective return of key frames and transition points 
in video [25, 42] could be used to subsample potential AFL and MSR navigation 
sequences, summarize flyby sequences for deep space operations or small bodies 
investigations, or identify the least-redundant images to transmit during EDL maneuvers. 
WATCH- style temporal change detection could certainly benefit a potential Titan lander, 
significantly improving science return at little mass or power cost. 

Aerial Vehicles 

Airships or balloons at Titan could cover orders of magnitude more terrain than that 
reachable by rovers. On Titan, an aerial perspective could resolve small-scale 
morphological features to investigate precipitation and liquid erosion processes [29], 
providing a dramatic counterpoint to our experience on Earth where with these 
phenomena dominate surface geology. Autonomous balloons at Venus could sample 



extensive vertical and horizontal profiles and address distinctive issues such as cloud 
structure, noble gas ratios and atmospheric superrotation. 

Most Titan and Venusian mission concepts presume significant wind currents and 
high ground-relative speeds [15]. This means that an aerobot could cross a wide area and 
collect far more data than could be returned at each command cycle. Autonomy 
requirements for these missions would therefore be comparable to or greater than those 
of surface vehicles, and autonomous data understanding could play a significant role to 
improve science return. Onboard analysis could compute science-relevant summary 
descriptions of Titan's terrain to draft summary maps, identify key frames for 
transmission to Earth, and possibly choose locations to deploy opportunistic sensors. 

Image analysis strategies used in ground-based platforms apply equally well to aerial 
imagery. Recent studies demonstrate automatic image classification based on 
quantitative texture attributes that correlate with morphological features [25] . These tests 
demonstrate fully automatic image analyses that are fast enough for onboard use and 
whose categorizations mirror expert interpretations of the scene. Mission planning and 
scheduling techniques also transfer to the aerial domain, permitting resource-cognizant 
reactions to science data [20]. Scientists may wish to target isolated features, maximize 
coverage of areas of interest, or reserve time for opportunistic data collection. Onboard 
data understanding provides the flexibility to pursue any or several of these goals while 
ensuring that the most valuable observations are included in the next downlink to Earth. 

Orbital Platforms Limited by Communication 

Imagery from the Mars Reconnaissance Orbiter (MRO) mission, in the form of Hi- 
RISE and CRISM data, has demonstrated the groundbreaking power of high-bandwidth 
orbital products such as high-resolution and hyperspectral images. Studies of fine 
morphological details reveal key mineral signatures that are highly localized or confined 
to a narrow layer of the geologic record [3, 14, 33]. These data products can assist in situ 
missions by resolving terrain features to identify landing areas and navigation hazards, 
and eventually by corroborating science measurements from the surface. However, 
innovation would be required to bring these benefits to future missions because the 
incremental upgrades planned for the deep space network will not support commensurate 
data rates to the outer planets. 

Onboard science data analysis offers a promising path to relieving this bottleneck. 
Automatic onboard classification can interpret hyperspectral data [38, 37, 4, 38, 31, 32] 
or prioritize key images to enable fast reconnaissance of planetary surfaces. It has already 
proven particularly valuable for terrestrial orbital imagery and hyperspectral imagery in 
particular [13]. 

Spacecraft equipped with change detection can follow up opportunistically for 
dynamic phenomena such as volcanism. The EO-1 sciencecraft has identified volcanic 
eruptions and triggered followup observations without direct human intervention [13]. 
Algorithms for onboard detection of volcanism have also been demonstrated for Io and 
Enceladus images [2], Many pushbroom imagers such as THEMIS, CRISM, or the IR 
imager considered for a potential Europa mission normally operate in a binned reduced- 
resolution mode due to bandwidth limits. Onboard analysis of data from pushbroom 
cameras can opportunistically apply full-resolution targeting to detected changes, areas of 
interest or compositional anomalies [39] . 



It is worth noting that onboard detection permits simultaneous, coordinated responses 
by multiple spacecraft for multimodal observations of transient phenomena. Earth 
observing assets have already been networked in this way, with observations by wide 
coverage assets that trigger followup observations by other assets with higher resolution 
or complementary modality [13]. As more assets are available for space science, 
enabling these sorts of synergistic observations or more pre-planned simultaneous 
observations will become of greater importance. 

Recommendations 

We close with recommendations for technology development and mission planning. 
It bears repeating that many of the technologies described above were developed under 
NASA programs that are no longer solicited. Building on the successes of initial efforts 
will require a greater resource commitment to develop new technologies and cultivate a 
pool of technologist talent in the area. In particular, growing funds for AISRP and 
ASTEP programs would sustain these recent advances. 

It is equally important to recognize onboard data analysis in early mission planning. 
To date the infusion of onboard data analysis has occurred through extended missions. 
We advocate planning explicitly for onboard science in advance and allocating resources 
to integrate these techniques into primary missions . Onboard analysis could yield far 
greater benefits if the capability informs early instrument selection and software design 
decisions. 

Automatic onboard data understanding can improve the science return of long range 
(over-fhe-horizon) traverses. The remote "field geologist" that surveys kilometers or tens 
of kilometers over the course of a mission can address significant science questions, and 
onboard data understanding can alleviate bandwidth constraints to make these operational 
modes feasible for outer-planets missions. Investments at the margin in the form of 
long-duration power, durable platforms, and long-lifespan instruments can enable wholly 
new survey-style investigations and yield profound improvements in science return. 

Additionally, we emphasize that bandwidth constraints need not preclude 
hyperspectral or high-resolution instruments . NASA's current plans for the Deep Space 
Network have charted a conservative upgrade path. Onboard data understanding and 
spacecraft autonomy can reduce the reliance on a constant high-volume link to Earth. 
The popularity of the MER panoramas underscores high-resolution imagery's potential to 
engage public participation and create a sense of virtual presence. Autonomy can help 
meet these rising expectations in the future. 

Finally, where margins permit, expanded CPU capability and especially mass memory 
can enhance science return through application of these techniques. Mars experience has 
shown that the software can be improved as the mission proceeds. While effective in the 
case of MER and EO-1, upgrades must overcome programmatic and institutional 
obstacles since they fit within existing flight software, computing resources and 
operations protocols. In the future we can catalyze adoption of onboard analysis with 
auxiliary computing resources and software design considerations. Large onboard 
memory caches can store data for later analysis at little cost in mass or power. These 
resources can benefit missions throughout the spacecraft lifespan as algorithms continue 
to improve. 
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